fetch_csv

by: qtmspin, 9 years ago


Super videos!  Well, done, clear and concise, learned a ton in a short time.

I'm using data from a csv that is similar to your sentiment data in structure.  I was able to copy your code to read that data.  My code keeps reading values from previous dates.  When I look at the fetched values it's using, the datetime values are incorrect.

Is it necessary to read a csv into variables to process like this post does?  
https://www.quantopian.com/posts/example-of-prefilling-data-when-using-fetcher-to-warm-up-algo

The returns on your sentiment tutorial looked great, did you find any errors, usually when I have returns like that something is wrong! ;)

ie
On 3/8 IPHI position is buy, when it should be 'undefined'.

position symbol date
buy IPHI 3/7/2014
buy BIIB 3/7/2014
buy CHTR 3/8/2014

code:
    try:
        for s in data:
            if 'position' in data[s]:
                pos_value = data[s]['position']




You must be logged in to post. Please login or register an account.



"Warming up" a strategy is only necessary if you need historical data at your starting point, as far as I know. This should only really be the case when employing live, so you don't have to wait, say, 100 days, to fill in your values.

I am seeing dates of what appears to maybe be "month day year" in your reports. That might be the beginning of the problem. With something like 3-7-2014, the machine has a hard time figuring out if thats mm-dd-yyyy or dd-mm-yyyy.

The sentiment signals did very well without shorting. When shorting, the returns were great, but leverage sometimes even hit 5x, so that'd need to be fixed to keep leverage down, and then returns would come down with it.. I believe in the past Quantopian allowed up to 3x leverage, but now its 1.05 max.

-Harrison 9 years ago

You must be logged in to post. Please login or register an account.


I know what the problem is now.  It's the difference in our data.  Your data starts w/ a small number of symbols and grows in symbols (always including previous symbols).  My data doesn't always include signals for symbols.  So somewhere fetch_csv is making up values to fill the gaps.  Since I don't yet know how to fix this w/ fetch_csv, I will make a global variable and dump the df into it, then use that data.  

ie your data:
2012-10-21 13:00:00,AAPL,-3
2012-10-22 13:00:00,AAPL,-3
2012-10-22 13:00:00,MSFT,2
2012-10-23 13:00:00,AAPL,-3
2012-10-23 13:00:00,MSFT,1
2012-10-23 13:00:00,GOOG,-3

my data (IPHI and BIIB) are not present in 3/8 data, so the data from 3/7 is being used
buy IPHI 3/7/2014
buy BIIB 3/7/2014
buy CHTR 3/8/2014

-qtmspin 9 years ago

You must be logged in to post. Please login or register an account.


Ah, nice catch. I believe in my code I actually searched to see if there even was an update to the company, but maybe that was not necessary if it fills forward.

-Harrison 9 years ago

You must be logged in to post. Please login or register an account.